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TO ALL WHOM IT MAY CONCERN : 

BE IT KNOWN THAT WE, TAKASHI KITAGUCHI, a 
citizen of Japan residing at Kanagawa, Japan, YASUHIRO 
SATO, a citizen of Japan residing at Kanagawa, Japan 
and NORIHIKO MURATA, a citizen of Japan residing at 
Kanagawa, Japan have invented certain new and useful 
improvements in 

APPARATUS AND METHOD FOR CORRECTION OF A DEVIATION OF 

DIGITAL CAMERA 

of which the following is a specification:- 
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1 BACKGROUND OF THE INVENTION 

( 1 ) Field of the Invention 

The present invention relates to an 
apparatus and a method for correction of a deviation of an 
5 imaging sensor of a digital camera which may be produced 
by a slight oscillation of the human hand in taking a 
photograph using the digital camera. 

( 2 ) Description of the Related Art 
Recently, the rapid growth of digital 

10 cameras for business uses as well as for personal uses has 
been experienced. Developments of digital cameras having 
a small size and a light weight with low cost will be 
increased. With the developments of such digital cameras, 
attention will be focused on correction of a deviation of 

15 a digital camera caused by a slight oscillation of the 
human hand when taking a photograph with the digital 
camera. 

If an imaging sensor of the digital camera 
deviates from a reference position due to the oscillation 

20 of the operator, a dim image will be reproduced by the 

digital camera. If the deviation of the imaging sensor is 
not corrected, it is difficult for the digital camera to 
provide good quality of a reproduced image. Therefore, 
there is a demand for a digital camera which is capable of 

25 automatically correcting a deviation of the imaging sensor 
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1 which may be produced by a slight oscillation of the 

operator who takes a photograph with the digital camera. 

In the fields of video cameras, techniques 
for correction of an image deviation of the video camera 

5 are known. For example, Japanese Laid-Open Patent 

Application No. 64-78581 discloses a video camera deviation 
correcting device. This deviation correcting device 
includes a frame memory which stores an image signal. An 
image deviation caused by a camera deviation is detected 

10 from a difference in image data between two frames in a 

sequence of continuously processed image frames. When an 
image deviation is detected, the image data is processed 
with the frame memory by controlling the read/write timing 
of the frame memory so as to correct the image deviation. 

15 In the device of the above publication, an 

acceleration sensor is provided to detect a camera motion. 
However, there is provided no moving mechanism which moves 
the imaging sensor based on the detected motion, so as to 
cancel the camera deviation having caused the image 

20 deviation. 

Japanese Laid-Open Patent Application No. 
2-103023 discloses a video camera deviation correcting 
device. The deviation correcting device includes 
horizontal and vertical line sensors which detect an image 
25 deviation caused by a camera deviation. Such an image 



deviation is detected from a difference in image data 
between two frames in a sequence of continuously processed 
image frames. When an image deviation is detected,, the 
imaging sensor is moved on a plane perpendicular to the 
optical axis of the video camera lens by a moving 
mechanism so as to correct the image deviation. 

Japanese Laid-Open Patent Application No. 
6-46322 discloses an imaging apparatus having a deviation 
correcting function. The deviation correcting function of 
the imaging apparatus is applied to a video camera. A 
camera motion is detected by a vibration sensor. When a 
camera motion is detected, the imaging sensor is moved on 
a plane perpendicular to the optical axis of the video 
camera lens by a moving mechanism such that the center of 
the imaging sensor is aligned with the optical axis of the 
video camera lens. 

The conventional techniques of the above 
publications are applied to the video cameras in which a 
sequence of image frames is continuously acquired and 
processed. However, it is very difficult to directly 
apply the video camera techniques to digital cameras in 
which a single frame of image data is acquired in an image 
acquisition time. In the case of digital cameras, it is 
necessary to correct a deviation of the imaging sensor of 
the digital camera within an image acquisition time for a 



single frame, in order to eliminate the degradation of 
quality of a reproduced image due to a slight oscillation 
of the operator. 

Further, in the conventional techniques of 
the above publications, there is no teaching about how to 
detect the magnitude and the direction of an imaging 
sensor deviation with a required level of accuracy, and 
how to move the imaging sensor based on the detection 
result in an effective manner so as to cancel the imaging 
sensor deviation. It is desirable to provide such new 
techniques for digital cameras, in order to. incorporate 
therein the functions to effectively correct a deviation 
of the imaging sensor and to reliably prevent the 
degradation of quality of a reproduced image due to a 
slight oscillation of the operator in taking a photograph 
with the digital camera. 

SUMMARY OF THE INVENTION 

An object of the present invention is to 
provide a novel and useful deviation correcting apparatus 
and method in which the above-described problems are 
eliminated. 

Another object of the present invention is 
to provide a digital camera deviation correcting apparatus 
which reliably prevents the degradation of quality of a 



reproduced image of the digital camera due to a slight 
oscillation of the operator by correcting a deviation of 
the imaging sensor of the digital camera in an effective 
manner through an accurate detection of the imaging sensor 
deviation and a controlled movement of the imaging sensor 
based on the detection result. 

Still another object of the present 
invention is to provide a digital camera deviation 
correcting method which reliably prevents the degradation 
of quality of a reproduced image of the digital camera due 
to a slight oscillation of the operator by correcting a 
deviation of the imaging sensor of the digital camera in 
an effective manner through an accurate detection of the 
imaging sensor deviation and a controlled movement of the 
imaging sensor based on the detection result. 

The above-mentioned objects of the present 
invention are achieved by an apparatus for correcting a 
deviation of an imaging sensor of a digital camera in 
which an image of an object or a scene is formed on an 
image plane of the imaging sensor so that the imaging 
sensor outputs an image signal, which comprises: a 
rotation detecting unit which detects a quantity of 
rotation of the digital camera causing a deviation of the 
imaging sensor from a reference position to occur; a 
positional angle calculating unit which calculates a 



change of a positional angle of the imaging sensor based 
on the rotation quantity detected by the rotation 
detecting unit; a target vector calculating unit which 
calculates a target vector based on the positional angle 
change calculated by the positional angle calculating 
unit, the target vector describing a magnitude and a 
direction of an inverse movement of the imaging sensor 
needed to reach the reference position and cancel the 
deviation; and a movement control unit which controls 
movement of the imaging sensor based on the target vector 
calculated by the target vector calculating unit, so that 
the imaging sensor is moved back to the reference position 
thus correcting the deviation, wherein the calculation of 
the target vector and the movement of the imaging sensor 
are executed within an image acquisition time for a single 
frame of the image signal. 

The above-mentioned objects of the present 
invention are achieved by a method of correcting a 
deviation of an imaging sensor of a digital camera in 
which an image of an object or a scene is formed on an 
image plane of the imaging sensor so that the imaging 
sensor outputs an image signal, comprising the steps of: 
detecting a quantity of rotation of the digital camera 
causing a deviation of the imaging sensor from a reference 
position to occur; calculating a change of a positional 




1 angle of the imaging sensor based on the detected rotation 
quantity; calculating a target vector based on the 
calculated positional angle change, the target vector 
describing a magnitude and a direction of an inverse 

5 movement of the imaging sensor needed to reach the 
reference position and cancel the deviation; and 
controlling movement of the imaging sensor based on the 
calculated target vector, so that the imaging sensor is 
moved back to the reference position thus correcting the 

10 deviation, wherein the calculation of the target vector 

and the movement of the imaging sensor are executed within 
an image acquisition time for a single frame of the image 
signal. 

In the digital camera deviation correcting 
15 apparatus and method of the present invention, a quantity 
of rotation and/or a quantity of translation of the 
digital camera, causing a deviation of the imaging sensor 
from the reference position to occur, is detected by a 
rotation detecting unit and/or a translation detecting 
20 unit. A change of the positional angle of the imaging 
sensor is calculated based on the detected rotation 
quantity. A change of the quantity of translation of the 
imaging sensor is calculated based on the detected 
translation quantity. A target vector is calculated based 
25 on the calculated positional angle change and the 



calculated translation quantity change, the target vector 
describing a magnitude and a direction of an inverse 
movement of the imaging sensor needed to reach the 
reference position and cancel the deviation. Movement of 
the imaging sensor is controlled based on the calculated 
target vector, so that the imaging sensor is moved back to 
the reference position thus correcting the deviation. The 
calculation of the target vector and the movement of the 
imaging sensor are executed within an image acquisition 
time for a single frame of the image signal. It is 
possible for the digital camera incorporating the 
principles of the present invention to provide good 
quality of a reproduced image as the degradation of 
quality of a reproduced image due to a slight oscillation 
of the operator is effectively prevented. It is possible 
that the apparatus and the method of the present invention 
effectively corrects a deviation of the imaging sensor of 
the digital camera through an accurate detection of the 
imaging sensor deviation and a controlled movement of the 
imaging sensor based on the detection result. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Other objects, features and advantages of 
the present invention will become more apparent from the 
following detailed description when read in conjunction 
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with the accompanying drawings in which: 

FIG. 1A is a block diagram of one 
embodiment: of a digital camera deviation correcting 
apparatus of the present invention; 

FIG. IB is a flowchart for explaining one 
embodiment of a digital camera deviation correcting method 
of the present invention; 

FIG. 2A is a block diagram of another 
embodiment of the digital camera deviation correcting 
apparatus of the present invention; 

FIG. 2B is a flowchart for explaining 
another embodiment of the digital camera deviation 
correcting method of the present invention; 

F*IG. 3 is a diagram for explaining a camera 
coordinate system and a world coordinate system used to 
execute a positional angle calculation in the digital 
camera deviation correcting apparatus of the present 
invention; 

FIG. 4 is a diagram for explaining the 
principles of a target vector calculation and an imaging 
sensor movement executed in the digital camera deviation 
correcting apparatus of the present invention; 

FIG. 5 is a perspective view of one 
embodiment of an imaging sensor movement mechanism 
applicable to the digital camera deviation correcting 



apparatus of the present invention; 

FIG. 6 is a top view of the imaging sensor 
movement mechanism of FIG. 5 when looking along a vertical 
rotation axis toward the origin of a camera coordinate 
system; 

FIG. 7 is a side view of the imaging sensor 
movement mechanism of FIG. 5 when looking along an optical 
axis of the digital camera toward the origin of the camera 
coordinate system; 

FIG. 8 is a side view of another embodiment 
of the imaging sensor movement mechanism applicable to the 
digital camera deviation correcting apparatus of the 
present invention; 

FIG. 9 is ,a diaigram for explaining the 
principles of the digital camera deviation correcting 
apparatus with the imaging sensor movement mechanism of 
FIG. 8 being incorporated therein; 

FIG. 10 is a perspective view of one 
embodiment of a translation detecting device applicable to 
the digital camera deviation correcting apparatus of the 
present invention; and 

FIG. 11 is a perspective view of one 
embodiment of a translation transmitting device applicable 
to the digital camera deviation correcting apparatus of 
the present invention . 
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1 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

A description will now be given of the 
preferred embodiments of the present invention with 
reference to the accompanying drawings. 

5 FIG. 1A shows a digital camera deviation 

correcting apparatus embodying the present invention. 

In a digital camera to which one embodiment 
of the present invention is applied, an image of an object 
or a scene is formed on an image plane of an imaging 

10 sensor so that the imaging sensor outputs a digital image 
signal. The digital camera deviation correcting apparatus 
of the present embodiment is provided for correcting a 
deviation of the imaging sensor of the digital camera 
which may be produced by a slight oscillation of the human 

15 hand in taking a photograph using the digital camera. 

As shown in FIG. 1A, the digital camera 
deviation correcting apparatus of the present embodiment 
includes a rotation detecting unit 10 which detects a 
quantity of rotation of the digital camera which causes a 

20 deviation of an imaging sensor 9 from a reference position 
to occur. The imaging sensor 9 is comprised of a CCD 
(charge-coupled device) or a CMOS (complementary metal 
oxide semiconductor) device. The rotation detecting unit 
10 may include a set of acceleration sensors provided to 

25 output signals indicative of accelerations of the digital 
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camera along an X axis, a Y axis and a Z axis of a world 
coordinate system, and a set of magnetic sensors provided 
to output signals indicative of magnetic fields of the 
digital camera along the X axis, the Y axis and the Z axis 
of the world coordinate system. 

A positional angle calculating unit 12 
calculates a change of a positional angle of the imaging 
sensor 9 based on the rotation quantity detected by the 
rotation detecting unit 10. The positional angle 
calculating unit 12 may include a set of analog-to-digital 
converters and a microprocessor with a memory. A target 
vector calculating unit 14 calculates a target vector 
based on the positional angle change calculated by the 
positional angle calculating unit 12, the target vector 
describing a magnitude and a direction of an inverse 
movement of the imaging sensor 9 needed to reach the 
reference position and cancel the deviation. The target 
vector calculating unit 14 may include a set of 
digital-to-analog converters and a microprocessor with a 
memory . 

In the digital camera deviation correcting 
apparatus of FIG. 1A, a drive control circuit 16 drives a 
displacement transmitting element 18 connected to the 
imaging sensor 9, based on the target vector calculated by 
the target vector calculating unit 14. The imaging sensor 
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9 

9 is moved in the inverse direction by a given 
displacement by the displacement transmitting element 18. 
During the movement of the imaging sensor 9, the resulting 
displacement of the imaging sensor 9 given by the 
displacement transmitting element 18 is detected by a 
displacement detecting element 20 connected to the imaging 
sensor 9 . A detection signal output by the displacement 
detecting element 20 is supplied to the drive control 
circuit 16. A closed loop control of the imaging sensor 
movement is carried out. Based on the detection signal 
supplied, the drive control circuit 16 continues to drive 
the displacement transmitting element 18 until the imaging 
sensor 9 reaches the reference position. The drive 
control circuit 16, the displacement transmitting element 
18 and the displacement detecting element 20 constitute a 
movement control unit which controls movement of the 
imaging sensor 9 so that the imaging sensor 9 is moved 
back to the reference position. Hence, the deviation of 
the imaging sensor 9 from the reference position is 
corrected by the movement control unit of the present 
embodiment . 

The displacement transmitting element 18 is 
comprised of a set of piezoelectric elements which change 
dimensions so as to transmit displacement to the imaging 
sensor 9 based on a drive voltage applied thereto. The 



displacement detecting element 20 is comprised of a set of 
piezoelectric elements which produce a voltage output to 
the drive control circuit 16 when stressed by the imaging 
sensor 9. 

In the digital camera deviation correcting 
apparatus of the present invention, the closed loop 
control of the imaging sensor movement is not necessarily- 
required • An open loop control of the imaging sensor 
movement may be applied to the movement control unit of 
the digital camera deviation correcting apparatus. In 
such a case, the displacement detecting element 20 and the 
feedback line to the drive control circuit 16 as in the 
apparatus of FIG. 1A may be omitted. In any case, the 
deviation of the imaging sensor 9 caused by a slight 
oscillation of the operator can be corrected by the 
movement control unit. 

It is possible for the digital camera v 
deviation correcting apparatus of the above-described 
embodiment to provide good quality of a reproduced image 
as the degradation of quality of a reproduced image due to 
a slight oscillation of the operator is reliably 
prevented. It is possible to effectively correct a 
deviation of the imaging sensor 9 of the digital camera 
through an accurate detection of the imaging sensor 
deviation and a controlled movement of the imaging sensor 
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1 based on the detection result. 

In the digital camera deviation correcting 
apparatus of the present embodiment, the calculation of 
the target vector and the movement of the imaging sensor 

5 are executed within an image acquisition time for a single 
frame of the image signal, which will be described later. 

FIG. IB is a flowchart for explaining a 
digital camera deviation correcting method embodying the 
present invention. The procedures of FIG. IB are executed 

10 by the digital camera deviation correcting apparatus of 
FIG. 1A. 

As shown in FIG. IB, at a start, the 
digital camera is set in a waiting condition. Step SI 
determines whether a power switch (not shown) of the 

15 digital camera is turned ON. When the power switch is* 
turned ON, step S2 determines whether a release button 
(not shown) of the digital camera is set at a half 
position. When the release button is set at the half 
position, it is determined that the function of correction 

20 of a deviation of the digital camera is allowed to start. 

When the result at the step S2 is 
affirmative, step S3 carries out the digital camera 
deviation correcting procedures with the digital camera 
deviation correcting apparatus of FIG. 1A. Namely, in the 

25 step S3, the rotation detection, the positional angle 



calculation, the target vector calculation and the imaging 
sensor movement are executed by the elements of the 
digital camera deviation correcting apparatus of FIG. 1A. 
After the step S3 is performed, step S4 determines whether 
an image acquisition time for a single frame of image data 
has elapsed. Many digital cameras are adapted .to use the 
NTSC standard, and the image acquisition time for one 
frame is normally 1/30 seconds. It is necessary to 
correct a deviation of the imaging sensor 9 of the digital 
camera within the image acquisition time for one frame. 
When the image* acquisition time has not yet elapsed, the 
step S3 is repeated. When the image acquisition time has 
already elapsed, the step S2 is repeated. 

When the result at the step S2 is negative, 
it is determined that the function of correction of a 
deviation of the digital camera is not allowed to start. 
Step S5 determines whether the power switch is turned OFF. 
When the power switch is turned OFF, the procedures of 
FIG. IB terminate. When the result at the step S5 is 
negative (the power switch is ON), the step S2 is 
repeated. 

According to the digital camera deviation 
correcting method of FIG. IB, the digital camera deviation 
correcting procedures in the step S3 are carried out 
before the image acquisition of a single frame of image 
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1 data is complete. It is possible for the digital camera 
deviation correcting method of the above-described 
embodiment to provide good quality of a reproduced image 
as the degradation of quality of a reproduced image due to 

5 a slight oscillation of the operator is reliably 

prevented. It is possible to effectively correct a 
deviation of the imaging sensor 9 of the digital camera 
through an accurate detection of the imaging sensor 
deviation and a controlled movement of the imaging sensor 

10 based on the detection result. 

FIG. 3 shows a camera coordinate system and 
a world coordinate system used to execute the positional 
angle calculation in the digital camera deviation 
correcting apparatus of the present embodiment. 

15 As shown in FIG. 3, the digital camera to 

which one embodiment of the present invention is applied 
generally has a main body 1 and an optical system (camera 
lens) 2. A camera coordinate system (x, y, z) and a world 
coordinate system (X, Y, Z) are selected as shown in FIG. 

20 3. Suppose that, initially, the digital camera was in a 
reference position, in the sense that the center of the 
digital camera (or the center of the image plane of the 
imaging sensor therein) was at the origin of the world 
coordinate system, and all the axes of both the systems 

25 were aligned. 
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1 The main body 1 of the digital camera is 

fixed to the camera coordinate system. An optical axis of 
the optical system 2 is aligned with the z axis of the 
camera coordinate system, which was initially aligned with 

5 the Z axis of the world coordinate system. The Y axis of 
the world coordinate system is taken to accord with the 
direction of gravity. The Z axis is taken to accord with 
the direction of the north pole of the earth. The X axis 
of the world coordinate system is perpendicular to both 

10 the Y axis and the Z axis. 

As shown in FIG. 3, in the digital camera, 
acceleration sensors 3, 4 and 5 are provided to output 
signals indicative of accelerations of the digital camera 
along the X axis, the Y axis and the Z axis, respectively. 

15 Further, magnetic sensors 6, 7 and 8 are provided to 
output signals indicative of magnetic fields of the 
digital camera along the X axis, the Y axis and the Z 
axis, respectively. The acceleration sensors 3, 4 and 5 
and the magnetic sensors 6, 7 and 8 constitute the 

20 rotation detecting unit of the digital camera deviation 
correcting apparatus of the present embodiment. 

Assume that the digital camera was 
initially in the reference position at a time t, and a 
deviation of the digital camera (or a deviation of the 

25 imaging sensor therein) from the reference position is 
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produced at: a following time (t+1) by a slight oscillation 
of the human hand in taking a photograph using the digital 
camera. In this assumption, as the deviating movement of 
the digital camera is considerably small in quantity, it 
is achieved only by rotation with no translation taking 
place. As a result of the above deviation, the digital 
camera is rotated from the reference position about the Y 
axis by a rotation angle 6y(t), about the X axis by a 
rotation angle 6x(t), and about the Z axis by a rotation 
angle 9z(t) to a new position. Specifically, the 
assumption is that the deviating movement of the digital 
camera from the reference position to the new position is 
achieved by three rotation steps: rotation of any point of 
the digital camera about the Y axis, rotation of the point 
of the digital camera about the X axis, and rotation of 
the point of the digital camera about the Z axis, in this 
order. Generally, the application of these rotation 
transformations can be represented by rotation matrices 
Ry(t:), Rx(t) and Rz(t), and the application of the inverse 
rotation transformations can be represented by the inverse 
rotation matrices Ry" 1 (t), Rx _1 (t) and Rz _1 (t). 

In addition, assume that, at the time t, 
the output signals of the acceleration sensors 3, 4 and 5 
were (Ax(t), Ay(t), Az(t)) (which is represented by an 
acceleration matrix [A(t)]), and the output signals of the 
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20 



magnetic sensors 6, 7 and 8 were (Mx(t), My(t), Mz(t)) 
(which is represented by a magnetic field matrix [M(t)])« 

As the digital camera was initially in the 
reference position at the time t, the following equation 
can be obtained. 

0 

Ry( t)Rx( t )Rz( t) [A( t )] - 1 

0 

From the above equation, the rotation angles 0x(t) and 
0z(t) of the imaging sensor are determined as follows. 

Gx(t) = -sin" 1 (Az(t)), 
Gz(t) = sin" 1 (Ax(t)/cos 9x(t)) 



(1) 



Further, 



Mx f 
My' 
Mz ' 
0 

My" 
Mz" 



Rx(t)Rz(t)[M(t)] / 



= Ry(t) 



Mx f 
My 1 
Mz' 
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From the above equations, the rotation angle 6y(t) of the 
imaging sensor is determined as follows, 

0y(t) = sin" 1 {Mx7/(Mx' 2 + Mz' 2 )} 

(2) 

The application of the rotation transformations at the 
time (t+1) can be represented as follows, 



R r (t+1) = {Ry(t)Rx(t)Rz(t)}" 1 Ry(t+l)Ry(t+l)Rz(-t+l) 

(3) 

From the above equation, a change of the positional angles 
of the imaging sensor of the digital camera between the 
initial time t and the following time (t+1) can be 
determined. 

FIG. 4 shows the principles of the target 
vector calculation and the imaging sensor movement 
executed in the digital camera deviation correcting 
apparatus of FIG. 1A. 

In FIG. 4, reference numeral 9 indicates 
the imaging sensor of the digital camera. The center of 
the imaging sensor 9 is indicated by "0 1 " in FIG. 4. The 
solid line in FIG. 4 denotes the reference position of the 
imaging sensor 9 in the digital camera, and the z axis of 
the camera coordinate system (which accords with the axial 
direction of the optical system 2 of the digital camera) 
is aligned with the Z axis of the world coordinate system. 
The y axis of the camera coordinate system is 
perpendicular to the paper of FIG. 4. 

The imaging sensor 9 is comprises of a CCD. 
As shown in FIG. 4, a rotation transmitting device 9A is 
connected to the imaging sensor 9, and this rotation 
transmitting device 9A is comprised of a set of voice coil 
motors which apply rotation to the imaging sensor 9 about 
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1 each of the x axis, the y axis and the z axis of the 

camera coordinate system based on a drive voltage ( or the 
target vector) supplied to the motors. Further, a 
translation transmitting device 9B is connected to the 

5 imaging sensor 9, and this translation transmitting device 
9B is comprised of a set of piezoelectric elements which 
apply translation to the imaging sensor 9 along each of 
the x axis, the y axis and the z axis of the camera 
coordinate based on a drive voltage (or the target vector) 

10 supplied to the piezoelectric elements. The rotation 

transmitting device 9A and the translation transmitting 
device 9B are provided in the movement control unit of the 
digital camera deviation correcting apparatus of FIG. 1A. 
The dotted line in FIG. 4 indicates a 

15 deviation of the imaging sensor 9 from the reference 

position which is caused by a slight oscillation of the 
operator. For the sake of simplicity of description, 
suppose that the imaging sensor 9 at this time is rotated 
from the reference position (the solid line) to the 

20 deviating position (the dotted line) about the center "0" 
of the optical system 2 of the digital camera by a 
rotation angle 11 0y" due to the oscillation of the 
operator. 

In this example of FIG. 4, a change of a 
25 positional angle of the imaging sensor 9, calculated by 
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the positional angle calculating unit 12 of the digital 
camera deviation correcting apparatus of FIG. 1A, is equal 
to the rotation angle 11 6y" . As being apparent from FIG. 
4, in order to correct the deviation of the imaging sensor 
9, application of rotation to the imaging sensor 9 about 
the y axis of the camera coordinate system by a rotation 
angle "-0y n is needed to be effected by the movement 
control unit of the apparatus of FIG. 1A. The rotation 
transmitting device 9A is driven by the movement control 
unit so that the imaging sensor 9 is rotated about the y 
axis by the rotation angle "-Gy" through the rotation 
transmitting device 9A. 

In the example of FIG. 4, a distance 
between the center "O" of the optical system and the 
center "O t " of the imaging sensor 9 along the z axis is 
represented by "f" . This distance "f" is varying when a 
zooming function of the digital camera is performed, but 
the distance "f" can be calculated in accordance with the 
actual zooming amount when the zooming function is 
performed. The distance "f" can be considered the known 
quantity. In the example of FIG. 4, in addition to the 
rotation about the y axis, the center "0 ,M of the imaging 
sensor 9 is translated to the center of the deviating 
position along the z axis by "f •sin 2 9y" and along the x 
axis by "-f»sin 0y*cos 0y" due to the oscillation of the 
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1 operator. Hence, in order to correct the deviation of the 
imaging sensor 9 into the reference position (the solid 
line in FIG. 4), application of translation to the center 
of the imaging sensor 9 along the z axis by " -f*sin 2 6y" 

5 and along the x axis by " f*sin Gycos 9y" is needed to be 
effected by the movement control unit of the apparatus of 
FIG. 1A. The translation transmitting device 9B is driven 
by the movement control unit so that the center of the 
imaging sensor 9 is translated along the z axis by " - 

10 f*sin 2 0y M and along the x axis by "f*sin Gycos 0y" 
through the translation transmitting device 9B. 

Accordingly, in the example of FIG. 4, the 
target vector calculating unit 14 of the apparatus of FIG. 
1A calculates a target vector based on the positional 

15 angle change "Gy", the target vector describing a 

magnitude and a direction of an inverse movement of the 
imaging sensor 9 ( including both the above-mentioned 
rotation and the above-mentioned translation) needed to 
reach the reference position and cancel the deviation. 

20 The inverse movement of the imaging sensor 9 is controlled 
through the rotation transmitting device 9A and the 
translation transmitting device 9B based on the calculated 
target vector, so as to move the imaging sensor 9 from the 
deviating position (the dotted line) back to the reference 

25 position (the solid line). 
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1 In the above example of FIG. 4, the 

application of only the rotation with respect to the y 
axis has been considered. In addition, the application of 
only the translations with respect to the z axis and the x 

5 axis. The same principles can be extended to include the 
rotations with respect to the x axis and the z axis as 
well as the translation with respect to the y axis, so 
that the application of the rotation about each of the x 
axis, the y axis and the z axis as well as the application 

10 of the translation along each of the x axis, the y axis 

and the z axis is effected by the movement control unit of 
the apparatus of FIG. 1A. 

In a case in which a translational 
displacement of the imaging sensor 9 can be detected by 

15 the apparatus of FIG. 1A and a change of the positional 
angle "6" is very small, the approximation 0 = tan 9 can 
be utilized in order to carry out the target vector 
calculation and the imaging sensor movement in the digital 
camera deviation correcting apparatus of FIG. 1A. 

20 FIG. 2A shows another embodiment of the 

digital camera deviation correcting apparatus of the 
present invention. In FIG. 2A, the elements which are the 
same as corresponding elements in FIG. 1A are designated 
by the same reference numerals, and a description thereof 

25 will be omitted. 
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1 As shown in FIG. 2A, the digital camera 

deviation correcting apparatus of the present embodiment 
includes a translation detecting unit 11 and a translation 
quantity calculating unit 13, in addition to the elements 

5 of FIG. 1A. 

The translation detecting unit 11 detects a 
quantity of translation of the digital camera along each 
of the X axis, the Y axis and the Z axis of the world 
coordinate system, which causes a deviation of the imaging 

10 sensor 9 from the reference position to occur. The 
translation detecting unit 11 is comprised of the 
acceleration sensors 3, 4 and 5. 

The translation quantity calculating unit . 
13 is connected to the target vector calculating unit 14. 

15 The translation quantity calculating unit 13 calculates a 
change of a quantity of translation of the imaging sensor 
9 based on the translation quantity detected by the 
translation detecting unit 11. The translation quantity 
calculating unit 13 may include a set of analog-digital 

20 converters and a microprocessor with a memory. The 

accelerations of the digital camera along the X axis, the 
Y axis and the Z axis of the world coordinate system are 
provided by the output signals of the acceleration sensors 
3, 4 and 5. A translational velocity of the imaging 

25 sensor 9 along each axis is calculated by taking the 
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integral of each of the accelerations with respect to 
time. A translational displacement of the imaging sensor 
9 along each of the X axis, the Y axis and the Z axis of 
the world coordinate system can be calculated by taking 
the integral of each of the velocity components with 
respect to time. 

In the present embodiment, the target 
vector calculating unit 14 calculates a target vector 
based on the positional angle change calculated by the 
positional angle calculating unit 12 and on the 
translation quantity change calculated by the translation 
quantity calculating unit 13. 

FIG. 2B is a flowchart for explaining 
another embodiment of the digital camera deviation 
correcting method of the present invention. The 
procedures of FIG. 2B are executed by the digital camera 
deviation correcting apparatus of FIG. 2A. In FIG. 2B, 
the steps which are the same as corresponding steps in 
FIG. IB are designated by the same reference numerals, and 
a description thereof will be omitted. 

As shown in FIG. 2B, the digital camera 
deviation correcting procedures of the present embodiment 
are essentially the same as the procedures of FIG. IB 
except step S13. In the step S13, the rotation detection, 
the positional angle calculation, the translation 
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detection, the translation quantity calculation, the 
target vector calculation and the imaging sensor movement 
are executed by the elements of the digital camera 
deviation correcting apparatus of FIG. 2A. In addition to 
the operations of the step S3 shown in FIG. IB, the 
translation detection and the translation quantity 
calculation are carried out by the element 11 and the 
element 13 of FIG. 2A. After the step S13 is performed, 
step S4 determines whether an image acquisition time for a 
single frame of image data has elapsed. When the image 
acquisition time has not yet elapsed, the step S13 is 
repeated. When the image acquisition time has already 
elapsed, the step S2 is repeated. 

According to the digital camera deviation 
correcting method of FIG. 2B, the digital camera deviation 
correcting procedures in the step S13 are carried out 
before the image acquisition of a single frame of image 
data is complete. It is possible for the digital camera 
deviation correcting method of the above-described 
embodiment to provide good quality of a reproduced image 
as the degradation of quality of a reproduced image due to 
a slight oscillation of the operator is reliably 
prevented. It is possible to effectively correct a 
deviation of the imaging sensor 9 of the digital camera 
through a more accurate detection of the imaging sensor 
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deviation and a controlled rotational and translational 
movement of the imaging sensor based on the detection 
result. 

FIG. 5 shows one embodiment of an imaging 
sensor movement mechanism applicable to the digital camera 
deviation correcting apparatus of the present invention. 
FIG. 6 is a top view of the imaging sensor movement 
mechanism of FIG. 5 when looking along the y axis toward 
the origin of the camera coordinate system. FIG. 7 is a 
side view of the imaging sensor movement mechanism of FIG. 
5 when looking along the z axis toward the origin of the 
camera coordinate system. 

In the digital camera deviation correcting 
apparatus incorporating the imaging sensor movement 
mechanism of FIG. 5 therein, the acceleration sensors 3, 4 
and 5 and the magnetic sensors 6, 7 and 8, which 
constitute the rotation detecting unit 10 of the digital 
camera deviation correcting apparatus of the embodiment of 
FIG. 1A, are provided. The 3D (three-dimensional) 
rotation components of a change of the positional angle of 
the imaging sensor 9 from the reference position are 
detected, and the imaging sensor movement mechanism of 
FIG. 5 achieves an inverse rotation of the imaging sensor 
9 in the 3D manner so as to correct a deviation of the 
imaging sensor 9 and move the imaging sensor 9 back to the 
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1 reference position. 

As shown in FIG. 5, in the digital camera 
to which the present embodiment is applied, an image of an 
object or a scene is formed through a camera lens 21 onto 

5 an image plane of the imaging sensor 9 (comprised of the 
CCD) so that the imaging sensor 9 outputs a digital image 
signal. The digital camera deviation correcting apparatus 
of the present embodiment is provided for correcting a 
deviation of the imaging sensor 9 which may be produced by 

10 a slight oscillation of the human hand in taking a 
photograph using the digital camera. 

The camera coordinate system (x, y, z) is 
taken to the digital camera in a manner similar to that of 
FIG. 3. Preferably, the origin "O" of the camera 

15 coordinate system (x, y, z) accords with the center of the 
optical system (or the focal point of the camera lens 21). 
In the imaging sensor movement mechanism of FIG. 5, a set 
of motors 33, 34 and 35 are provided to respectively apply 
rotation to the imaging sensor 9 about the y axis, the x 

20 axis and the z axis of the camera coordinate system based 
on a target vector. FIG. 6 shows a connection of the 
motor 34 and the motor 35 which are held in the digital 
camera, and FIG. 7 shows a connection of the motor 33 and 
the motor 34 which are held in the digital camera. 

25 As shown in FIG. 6, the motor 35 has a 
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rotary shaft fixed to the center of the imaging sensor 9 
(the CCD), and the rotary shaft of the motor 35 is rotated 
about the z axis of the camera coordinate system. The 
motor 35 has a base secured to a first rectangular surface 
of an L-shaped member 36. The L-shaped member 36 includes 
a second rectangular surface which is perpendicular to the 
first rectangular surface and parallel to the yz plane. 
The motor 34 has a rotary shaft fixed to the second 
rectangular surface of the L-shaped member 36. The rotary 
shaft of the motor 34 is rotated about the x axis of the 
camera coordinate system. When the rotary shaft of the 
motor 34 is rotated, the imaging sensor 9 and the motor 35 
are rotated together about the x axis through the L-shaped 
member 3 6 . 

As shown in FIG. 7, the motor 34 has a base 
secured to a first rectangular surface of an L-shaped 
member 37. The L-shaped member 37 includes a second 
rectangular surface which is perpendicular to the first 
rectangular surface and parallel to the xz plane. The 
motor 33 has a rotary shaft fixed to the second 
rectangular surface of the L-shaped member 37. The motor 
33 has a base secured to the main body of the digital 
camera (which is not shown in FIG. 7). The rotary shaft 
of the motor 33 is rotated about the y axis of the camera 
coordinate system. When the rotary shaft of the motor 33 
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1 is rotated, the motor 34, the motor 35 and the L-shaped 
member 37 are rotated together about the y axis through 
the L-shaped member 37. 

Similar to the calculation of the 

5 positional angle change of FIG. 3, the digital camera was 
initially in the reference position at the time t, and the 
rotation angles 0x(t) and 8z(t) of the imaging sensor 9 
are determined in accordance with the above equations ( 1 ) . 
The rotation angle 6y(t) of the imaging sensor 9 is 

10 determined in accordance with the above equation ( 2 ) . 

The application of the rotation 
transformations at the time (t+1) can be represented by 
the above equation ( 3 ) . From the above equation ( 3 ) , a 
change of the positional angles of the imaging sensor 9 of 

15 the digital camera between the initial time t and the 
following time (t+1) can be determined. 

Assuming that the change of the positional 
angles of the imaging sensor 9 is determined as e^xCt+l), 
e^ft+l) and 9 t z(t+l) / the motors 34, 33 and 35 are 

20 controlled based on the target vector to respectively 

apply rotation to the imaging sensor 9 about the x axis by 
-0' t x(t+l) / about the y axis by -G^ytt+l) and about the z 
axis by -e^zft+l), so as to cancel the deviation of the 
imaging sensor 9 and move the imaging sensor 9 back to the 

25 reference position. 



- 34 - 



FIG. 8 shows another embodiment of the 
imaging sensor movement mechanism which is applicable to 
the digital camera deviation correcting apparatus of the 
present invention . 

In the digital camera deviation correcting 
apparatus incorporating the imaging sensor movement 
mechanism of FIG. 8 therein, a 2D (two-dimensional) gyro, 
which constitutes the rotation detecting unit 10 of the 
digital camera deviation correcting apparatus of the 
embodiment of FIG. 1A, is provided. The 2D gyro detects 
angular velocities of the imaging sensor 9 with respect to 
the x axis and the y axis of the camera coordinate system. 
The 2D rotation components of a change of the positional 
angle of the imaging sensor 9 from the reference position 
are detected, and the imaging sensor movement mechanism of 
FIG. 8 achieves an inverse translation of the imaging 
sensor 9 in the 2D manner so as to correct a deviation of 
the imaging sensor 9 and move the imaging sensor 9 back to 
the reference position. In this case, the rotational 
movement of the digital camera is considerably small, and 
it is achieved only by translation with no rotation taking 
place . 

FIG. 9 shows the principles of the digital 
camera deviation correcting apparatus with the imaging 
sensor movement mechanism of FIG. 8 being incorporated 



-therein . 

As shown in FIG. 9, in the digital camera 
to which the present embodiment is applied , an image of an 
object is formed through the camera lens 21 onto the image 
plane of the imaging sensor 9 (comprised of the CCD), The 
center of the image of the object on the image plane of 
the imaging sensor 9 is indicated by "0" in FIG. 9. The 
reference position (the solid line in FIG. 9) of the 
imaging sensor 9 is indicated by "PI" in FIG. 9. A 
distance between the center of the imaging sensor 9 and 
the center of the camera lens 21 is indicated by "f" in 
FIG. 9. 

Suppose that the digital camera is rotated 
by a rotation angle "6" with respect to the optical axis 
of the camera lens 21 (or the z axis of the camera 
coordinate system) due to a slight oscillation of the 
operator. The rotation angle "0" is very small. The 
image plane of the imaging sensor 9 is moved from the 
reference position "PI", to a deviating position "P2" (the 
dotted thin line in FIG. 9) due to the deviation of the 
imaging sensor 9. The center of the image of the object 
on the image plane of the imaging sensor 9 at the 
deviating position "P2" is indicated by "O'" in FIG. 9. 

In the example of FIG. 9, in order to 
correct the deviation of the imaging sensor 9, application 
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1 of translation to the center of the imaging sensor 9 by 
the distance d = f^tan 0. After the translation is 
applied, the imaging sensor 9 is moved from the deviating 
position "P2" to a corrected position "P3" (the dotted 

5 thick line in FIG. 9). As the rotation angle "0" is very 
small, the approximation 0 = tan 0 can be utilized. 
Hence, application of the translation to the center of the 
imaging sensor 9 by the distance d = f«0 is needed to be 
effected by the movement control unit of the apparatus of 

10 FIG. 1A. 

Assuming that the change of the positional 
angles of the imaging sensor 9 is calculated as 0**y(t+l) 
and ©"^(t+l), the application of translation to the 
imaging sensor 9 along the y axis by -f-O^yCt+l) and along 

15 the z alxis by -f • B^zi t+1 ) , so as to cancel the deviation 

of the imaging sensor 9 and move the imaging sensor 9 back 
to the reference position. The output signals of the 2D 
gyro are the angular velocities of the imaging sensor 9, 
and the change of the positional angles can be calculated 

20 by taking the integral of each of the velocity components 
from the time t to the time (t+1). 

In the embodiment of the imaging sensor 
movement mechanism of FIG. 8, a piezoelectric element 38 
is secured at one end to the base of the imaging sensor 9 

25 (the CCD). The piezoelectric element 38 is secured at the 
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1 other end to a first surface of an L-shaped member 39. 
The piezoelectric element 38 is provided to apply 
translation of the imaging sensor 9 along the y axis of 
the camera coordinate system. The L-shaped member 39 has 

5 a second surface which is perpendicular to the first 
surface and parallel to the y axis. A piezoelectric 
element 40 is secured at one end the second surface of the 
L-shaped member 39, and secured at the other end to the 
main body of the digital camera. The piezoelectric 

10 element 40 is provided to apply translation of the imaging 
sensor 9 along the x axis of the camera coordinate system. 
Accordingly, the imaging sensor movement mechanism of FIG. 
8 achieves an inverse translation of the imaging sensor 9 
in the 2D manner so as to correct a deviation of the 

15 imaging sensor 9 and move the imaging sensor 9 back to the 
reference position. 

FIG. 10 shows an embodiment of a 
translation detecting device which is applicable to the 
digital camera deviation correcting apparatus of the 

20 present invention. 

As shown in FIG. 10, a range finder 41 is 
provided on the digital camera, and this range finder 41 
measures distances of objects from the digital camera 
along three lines, indicated by the three arrows in FIG. 

25 10, which are all perpendicular to each other. The range 
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1 finder 41 acts as the translation detecting device which 

detects a quantity of translation of the digital camera in 
the digital camera deviation correcting apparatus of the 
present invention . 

5 In the digital camera deviation correcting 

apparatus with the translation detecting device 41 
incorporated therein, the acceleration sensors 3, 4 and 5 
and the magnetic sensors 6, 7 and 8 are also provided. 
The acceleration sensors 3, 4 and 5 and the magnetic 

10 sensors 6, 7 and 8 constitute the rotation detecting unit 
of the digital camera deviation correcting apparatus of 
the present embodiment. The positional angle calculation 
is carried out based on the output signals of these 
sensors 3 through 8 in the same manner as the above- 

15 described positional angle calculation with reference to 
FIG. 3. 

In the digital camera deviation correcting 
apparatus with the translation detecting device 41 
incorporated therein, a translation quantity calculation 
20 is carried out based on output signals of the range finder 
41 as follows. 

As described above, the range finder 41 
outputs signals indicative of distances between objects 
and the digital camera in the three directions. If three 
25 distant fixed objects in the three directions are given, 
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1 the range finder 41 outputs the signals indicating the 
distances of the fixed objects from the digital camera. 

Suppose that the range finder 41 has output 
the distance signals [dx(t), dy(t), dz(t)] at the initial 

5 time t and the distance signals [dx(t+l), dy(t+l), 

dz(t+l)] at the following time t+1. A change of quantity 
of translation of the imaging sensor of the digital camera 
which may be produced by a slight oscillation of the 
operator is represented by [ -( dx( t+1 ) -dx( t ) ) , dy(t+l)- 

10 dy(t), -(dz( t+1 )-dz( t) )] . Hence, if the positional angle 
change is not considered, the target vector describing a 
magnitude and a direction of an inverse translation of the 
imaging sensor 9 needed to reach the reference position 
f and cancel the deviation is represented by [dx( t+1 ) -dx( t ) , 

15 -(dy(t+l)-dy(t) ), dz ( t+1 ) -dz( t ) ] . 

In the digital camera deviation correcting 
apparatus with the translation detecting device 41 
incorporated therein, the target vector calculating unit 
calculates the target vector based on the calculated 

20 positional angle change and on the calculated translation 
quantity change as in the embodiment of FIG. 2A. 

FIG. 11 shows one embodiment of a 
translation transmitting device which is applicable to the 
digital camera deviation correcting apparatus of the 

25 present invention. 
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As shown in FIG. 11, piezoelectric elements 
42 , 43 and 44 are linked together such that the 
piezoelectric elements 42, 43 and 44 are aligned with the 
x axis, the y axis and the z axis of the camera coordinate 
system. One end of the piezoelectric element 42 is fixed 
to the main body of the digital camera. One end of the 
piezoelectric element 44 is fixed to the motor 33 of the 
imaging sensor movement mechanism of FIG. 5. By supplying 
a drive signal from the drive control circuit to each of 
the piezoelectric elements 42-44, each piezoelectric 
element achieves a translational movement of the imaging 
sensor 9 along one of the x axis, the y axis and the z 
axis of the camera coordinate system so as to cancel the 
deviation of the imaging sensor 9 . 

The piezoelectric elements 42, 43 and 44 
act as the translation transmitting device which achieves 
a translational movement of the imaging sensor 9 based on 
the calculated target vector in the digital camera 
deviation correcting apparatus of the present invention. 
By utilizing the translation transmitting device of FIG. 
11 and the rotation transmitting device of FIG. 5 in 
combination, the movement control unit in the digital 
camera deviation correcting apparatus of the present 
invention can be constituted. This movement control unit 
controls rotational and translational movements of the 
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imaging sensor 9 based on the calculated target vector, so 
that the imaging sensor 9 is moved back to the reference 
position thus correcting the deviation. 

Further, the present invention is not 
limited to the above-described embodiments, and variations 
and modifications may be made without departing from the 
scope of the present invention. 



